1. Field of the Invention
The present invention relates to an image pick-up apparatus for stereoscope for picking up parallax images for stereoscopic viewing. More particularly, the present invention relates to an image pick-up apparatus for picking up, without user intervention, parallax images that satisfy binocular fusion conditions and an image pick-up apparatus that lets a user know whether the binocular fusion conditions are satisfied.
2. Description of the Related Art
Some known stereoscopic image pick-up apparatuses capture parallax images from a plurality of view points using a plurality of cameras. In such a stereoscopic image pick-up apparatus, a plurality of cameras are mounted on tripod heads, and a user manually adjusts a spacing and a convergence angle between camera axes, based on his or her operational experience, depending on a scene to be captured, and image-captures the scene while actually observing the stereoscopic view on a monitor.
Such an experience-based operation is not efficient, and no good result is expected when there is no one available who has a good knowledge of picking up stereoscopic images.
Head-mounted displays and eyeglass-type displays have been developed today, and these displays let an image for the right eye to be selectively presented on the right eye and an image for the left eye to be selectively presented on the left eye. A stereoscopic image producing a depth perception is thus observed. A liquid-crystal display is combined with a lenticular sheet having a predetermined pitch or a mask with apertures and non-apertures formed in a predetermined pattern to impart a directivity to a light ray from the liquid-crystal display, and by making the directivity match the pattern of the image presented on the liquid-crystal display, the observer watches the image for the right eye on the right eye and the image for the left eye on the left eye. The observer thus enjoys the image presenting depth.
Conventionally, the image presented is typically acquired through a binocular-type camera having two lens sets. An apparatus, disclosed in Japanese Examined Patent Publication No. 8-27499 (telescopic television image pick-up apparatus), does not require two lens sets. This apparatus includes two liquid-crystal shutters, a total reflection mirror and a half mirror and picks up alternately left and right parallax images through a single lens set.
Regardless of the above binocular system or the system picking up alternately left and right images in a time division manner, the adjustment of a parallax between left and right images, namely, the adjustment of convergence is required during image-capturing. The convergence adjustment is typically manually performed.
The above binocular-type camera employs two lens sets, one for forming the image for the right eye and the other for forming the image for the left eye, and the user feels fatigue when observing a stereoscopic image or cannot to attain image fusion at all, if performance differences due to manufacturing error (for example, in magnification, deviation in optical axis, tint, brightness, distortion, field tilt and the like) are present between the two lenses. To make the two lens sets match in performance, the accuracy of components needs to be heightened. Adjustment is further required if the improved accuracy of components is still not sufficient enough. To compensate for performance difference, special means is used, for example, images are electronically corrected. When a zoom lens is used, the zoom variator operations of the left and right lens sets must be interlocked in a state with these performances factored in. This arrangement is costly and time consuming to manufacture, and is inadequate for mass production.
To observe the images picked up by the binocular-type camera, two monitors are typically required. The use of the two monitors is not practical. To record the two images, a special recording device is needed to synchronize the one image to the other. Without using it, the conversion of two image signals into one image signal is contemplated. To covert the two image signals into one image signal, however, a special converter is required to alternately display and record the left and right parallax images. Compared with a single lens camera, the binocular-type camera itself is bulky. The entire system of the image pick-up apparatus, in need of the above-described special devices, fails to achieve widespread commercial acceptance among users because of its bulkier, costlier and less mobile design.
The time-division camera proposed in the above-cited Japanese Examined Patent Publication No. 8-27499 combines the optical paths for left and right parallax images at a half mirror into one optical path to guide images to a single lens. When images are transmitted through or reflected from the half mirror before entering the lens, the quantity of light is reduced to half or smaller. The arrangement disclosed in the above-cited Japanese Examined Patent Publication No. 8-27499 theoretically presents a difference between the lengths of the optical paths of the left and right parallax images, suffering a magnification difference between the left and right images. This becomes a cause of the fatigue of the user when he or she observes the images picked-up, and as a result, the user cannot observe the images fused and cannot observe them in a stereoscopic view.
Accordingly, it is an object of the present invention to provide an image pick-up apparatus through which a user or viewer easily learns during image-picking up whether a main subject falls within a range of image fusion.
The object of the present invention is achieved by the image pick-up apparatus for capturing images for stereoscopic viewing, which comprises image pick-up means for picking up left and right parallax images of a main subject respectively for the left and right eyes, display means for displaying the left and right parallax images, picked up by the image pick-up means, line-of-sight detection means for detecting the lines of sight of the left and right eyes, looking the respective images displayed by the display means, and determining means for determining, based on the output of the line-of-sight detection means, whether the main subject falls within an image-fusible range. The apparatus detects the lines of sight of an observer who observes the display means on which the left parallax image and right parallax image are presented, and, instead of the observer, determines, based on the lines of sight, whether the subject falls within the image-fusible range.
In a preferred embodiment of the preset invention, the determination result provided by the determining means is reported to the user.
In a preferred embodiment of the present invention, an image-fusible range is defined by the inter-pupillary distance and the distance of distinct vision of a user.
In a preferred embodiment of the present invention, the image date of the parallax images is stored in a memory in response to the output of the determining means.
Since the detected lines of sight of the user are the ones that look toward the display parallax images, the data about the lines of sight need to be converted into those in the coordinate systems of the image pick-up means. In a preferred embodiment of the present invention, the line-of-sight detection means further comprises converting means for converting the left and right lines of sight of a user into left and right direction vectors that are respectively expressed in left and right coordinate systems of the image pick-up means, and coordinates calculator means for calculating the coordinate values of the crossing point of the left and rightdirection vectors in the world coordinate system.
In a preferred embodiment of the present invention, the image-fusible range is expressed according to a farthest position and a nearest position from the view point of the user in the direction of depth. In a preferred embodiment, the farthest position of the image-fusible range is set to be a point so that the horizontal distance between two second points on the left and right image planes of the image pick-up means corresponding to a first point (A) of the farthest position is substantially equal to the inter-pupillary distance of the user. The nearest position of the image-fusible range is set to be a point so that a position (Cxe2x80x2) where two points on the left and right image planes of the image pick-up means corresponding to a first point (C) of the nearest position, through perspective transformation based on the left and right view points of the user, look standing out to the user, is approximately equal to the point at the distance of distinct vision of the user.
It is another object of the present invention to provide an image pick-up apparatus that is compact, low-cost, features mobility and expandability, and presents a high-quality stereoscopic image.
It is yet another object of the present invention to reduce the work load of a user or viewer during image-picking up by automating the convergence control and to enable the user to capture a natural-looking stereoscopic image.
These objects are achieved by the image pick-up apparatus, which comprises a first optical system and a second optical system with a predetermined convergence angle made therebetween, a first electronic shutter and a second electronic shutter for electronically blocking the respective optical paths of the first and second optical systems, control means for driving the first and second electronic shutters in a time-division manner, optical path integrator means for integrating the optical paths of the first and second optical systems, a third optical system having the integrated optical path, a charge-coupled device for photoelectrically converting an optical image transmitted through the third optical system, reading means for reading the output of the charge-coupled device in a time-division manner in synchronization with time-division driving of the first and second electronic shutter by the control means, distance measuring means for measuring a distance to a subject, and adjusting means for adjusting the convergence angle between the first and second optical systems in accordance with the measured distance.
The image pick-up apparatus is further made compact by incorporating the optical path integrator means which comprises a prism arranged at the entrance of the optical path of the third optical system, a first mirror for deflecting the optical path of the first optical system toward the prism, a second mirror for deflecting the optical path of the second optical system toward the prism, and the adjusting means which comprises angle adjusting means for controlling the angles of pivot of the first and second mirrors.
In a preferred embodiment of the present invention, the angle adjusting means pivots the first and second mirrors in the same angles but in opposite directions.
A variety of methods may be used to measure the distance to the subject. More particularly, in a preferred embodiment of the present invention, the distance measuring means uses a triangulation method. In a preferred embodiment of the present invention, distance measurement is performed based on position information about the plurality of lenses used in the first and second optical systems.
The distance to the subject changes. It is necessary to obtain proper parallax images of the subject, the distance to which is changed.
To achieve this object, in a preferred embodiment of the present invention, the image pick-up apparatus further comprises detector means for detecting a change, in the distance to the subject, in excess of a predetermined value, and activating means for activating the adjusting means in response to the output of the detector means.
To achieve the same object, in a preferred embodiment of the present invention, the image pick-up apparatus further comprises detector means for detecting a predetermined number or larger number of occurrences of changes in the distance to the subject, each change in excess of a predetermined value, and activating means for activating the adjusting means in response to the output of the detector means.
In a preferred embodiment of the present invention, the image pick-up apparatus further comprises a camera main unit for processing an output signal from the charge-coupled device, a lens unit for driving the optical systems, and an interconnection unit for electrically connecting the camera main unit to the lens unit.
In a preferred embodiment of the present invention, in order to seek further the expandability and compact design, the image pick-up apparatus further comprises a camera mount for the camera main unit and a lens mount for the lens unit, wherein the lens unit is detachably mounted onto the camera main unit.